The present invention relates to a method of extracting a target object from an image sensed by an image sensing apparatus, a method of cutting out the object, a database structure used in extraction and a method of creating the database, and an image sensing apparatus or an image sensing system that can obtain object information using these methods. The present invention also relates to a storage medium which provides a program and data to the image sensing apparatus or image sensing system or stores the database.
As a technique for discriminating the presence/absence of a specific object in an image, or searching a database for an image including a specific object and extracting the image, a pattern recognition technique is used. Methods of applying a pattern recognition technique upon executing the pattern recognition include the following methods.
More specifically, in the first method, an image is segmented into a plurality of regions in advance and cutting processing is performed so that only a specific region to be recognized remains. Thereafter, similarity with a standard pattern is calculated using various methods.
In the second method, a template prepared in advance is scanned to calculate the degree of matching (correlation coefficient) at the respective positions to search for a position where the calculated value becomes equal to or larger than a predetermined threshold value (Japanese Patent Laid-Open No. 6-168331).
Furthermore, in the third method, upon creating an image database, regions of constituting elements and constituting element names in an image are input, so as to attain high-speed search for an image having a predetermined feature (Japanese Patent Laid-Open No. 5-242160).
However, in the first and second methods, since the position or size of a specific object in an image or the hue or the like that reflects the illumination condition is not known in advance, the following problems are posed.
First, since similarity must be calculated using a plurality of standard patterns (images representing identical objects having different sizes, positions, hues, and the like), a considerably large calculation amount and long calculation time are required.
Second, it is generally difficult to find and cut out a specific region having a feature close to that of a standard pattern for the same reason as in the first problem.
Third, the template size can be set in advance under only very limited image generation conditions. When the image generation conditions are not known, the same problem as the first problem is posed. Therefore, a very long calculation time is required for discriminating the presence/absence of a specific object, searching for an image including a specific object, and the like.
In the third method, in order to input regions of constituting elements and their names in an image, input interfaces such as a keyboard, mouse, and the like are required, and when a database of images actually sensed by an image sensing means is to be created, such search data must be created after the image sensing operation.
Furthermore, an application for searching a database of images sensed using an image sensing means for an image including an object intended to be generally the main object in the scene cannot be realized by conventional image processing methods that do not use any information upon image sensing.
As a general technique for extracting (cutting) an image, a chromakey technique using a specific color background, a videomat technique for generating a key signal by image processing (histogram processing, difference, differential processing, edge emphasis, edge tracking, and the like) (Television Society technical report, vol. 12, pp. 29-34, 1988), and the like are known.
As another apparatus for extracting a specific region from an image, in a technique disclosed in Japanese Patent Publication No. 6-9062, a differential value obtained by a spatial filter is binarized to detect a boundary line, connected regions broken up by the boundary line are labeled, and regions with an identical label are extracted.
A technique for performing image extraction based on the difference from the background image is a classical technique, and recently, Japanese Patent Laid-Open No. 4-216181 discloses a technique for extracting or detecting target objects in a plurality of specific regions in an image by setting a plurality of masks (=specific processing regions) in the difference data between background image and the image to be processed.
In a method associated with Japanese Patent Publication No. 7-16250, the distribution of probability of occurrence for the object to be extracted is obtained on the basis of the color-converted data of the current image including the background image, and the difference data between the lightness levels of the background image and the current image using a color model of the object to be extracted.
As one of techniques for extracting a specific object image by extracting the outer contour line of the object from an image, a so-called active contour method (M. Kass et al., xe2x80x9cSnakes: Active Contour Models,xe2x80x9d International Journal of Computer Vision, Vol. 1, pp. 321-331, 1987) is known.
In the above-mentioned technique, an initial contour which is appropriately set to surround an object moves and deforms (changes its shape), and finally converges to the outer shape of the object. In the active contour method, the following processing is typically performed. More specifically, a contour line shape u(s) that minimizes an evaluation function given by equation (1) below is calculated with respect to a contour line u(s)=(x(s), y(s)) expressed using a parameter s that describes the coordinates of each point:                     E        =                                            ∫              0              u                        ⁢                                          E                1                            ⁡                              (                                  V                  ⁡                                      (                    s                    )                                                  )                                              +                                    w              0                        ⁢                                          E                0                            ⁡                              (                                  V                  ⁡                                      (                    s                    )                                                  )                                      ⁢                          xe2x80x83                        ⁢                          ⅆ              s                                                          (        1        )                                          For          ⁢                      xe2x80x83                    ⁢                                    E              1                        ⁡                          (                              V                ⁡                                  (                  s                  )                                            )                                      =                                            α              ⁡                              (                s                )                                      ⁢                                          "LeftBracketingBar"                                                      ⅆ                    u                                                        ⅆ                    s                                                  "RightBracketingBar"                            2                                +                                    β              ⁡                              (                s                )                                      ⁢                                          "LeftBracketingBar"                                                                            ⅆ                      2                                        ⁢                    u                                                        ⅆ                                          s                      2                                                                      "RightBracketingBar"                            2                                                          (        2        )            xe2x80x83E0(V(s))=xe2x88x92|xcex94I(u(s))|2xe2x80x83xe2x80x83(3)
where I(u(s)) represents the luminance level on u(s), and xcex1(s), xcex2(s), and w0 are appropriately set by the user. In the technique (active contour method) for obtaining the contour line of a specific object by minimizing the above-mentioned evaluation function defined for a contour line, setting methods described in Japanese Patent laid-Open Nos. 6-138137, 6-251148, 6-282652, and the like are known as the setting method of an initial contour.
The chromakey technique cannot be used outdoors due to strict limitations on the background, and also suffers a problem of color omission. In the videomat technique, the user must accurately perform contour designation in units of pixels, thus requiring much labor and skill.
The technique using the difference from the background image cannot be normally applied when an image of only the background except for a specific object cannot be sensed (e.g., the object is huge), and the load on the user is heavy.
Since no image sensing conditions (camera parameters and external conditions such as illumination) are taken into consideration, discrimination errors of the region to be extracted from the difference data become very large unless the background image and the image including the object to be extracted are obtained under the same image sensing conditions and at the same fixed position. Also, the technique described in Japanese Patent Publication No. 7-16250 is not suitable for extracting an image of an unknown object since it requires a color model of the object to be extracted.
Of the initial contour setting methods of the above-mentioned active contour method, in Japanese Patent Laid-Open No. 6-138137, an object region in motion is detected on the basis of the inter-frame difference, and a contour line is detected on the basis of contour extraction (searching for the maximum gradient edge of a changed region) in the vicinity of the detected region. Therefore, this method cannot be applied to a still object in an arbitrary background.
In Japanese Patent Laid-Open No. 6-282652, feature points with a strong edge are extracted from an image, and points with higher evaluation values are selected from a set of feature points on the basis of the evaluation function, thereby setting the initial contour. In this case, the background image must be plain or image data that changes gradually.
Furthermore, as an example of the technique for optimizing the camera operation and the operation mode, in a method described in Japanese Patent Laid-Open No. 6-253197, the stop is set to obtain an appropriate average luminance upon sensing the background image. Thereafter, the current image is sensed using the same setting value, and the object image is extracted on the basis of difference image data therebetween.
On the other hand, as the degree of freedom in processing and modification of video information becomes higher along with the advance of digital signal processing, the internal processing of the image sensing means has seen a great change from relatively simple processing such as luminance level or color tone conversion, white-balance processing, quantization size conversion, and the like to one having an edge extraction function, and one having an image extraction function using a color component sequential growth method (Television Society technical report, Vol. 18, pp. 13-18, 1994).
However, since the methods that use difference data from an image of only the background do not consider any image taking conditions (camera parameters and external conditions such as illumination) except for the technique described in Japanese Patent Laid-Open No. 6-253197, discrimination errors of the region to be extracted from difference data become very large unless the background image and the image including the object to be extracted are obtained under the same image taking conditions and at the same fixed position.
On the other hand, the method described in Japanese Patent Publication No. 7-16250 is not suitable for extraction of an image of an unknown object since it requires a color model of the object to be extracted.
The method associated with Japanese Patent Laid-Open No. 6-253197 merely discloses a technique in which the setting value of the stop upon sensing the background image is used upon sensing an image including a specific object on the premises that the image sensing means is set at the same fixed position, and the same image sensing conditions as those upon sensing the image including only the background are used. In this method that gives priority to the image sensing conditions of the background image, the image quality of the object to be extracted, i.e., an image including a specific object is not normally guaranteed.
Furthermore, the chromakey method cannot be used outdoors due to serious limitations on the background and also suffers a problem of color omission.
Also, in the videomat method, the contour designation operation must be manually and accurately performed in units of pixels, thus requiring much labor and skill.
The method of detecting regions segmented by a boundary line by detecting the boundary line by differential calculations can hardly be applied to an object having a complex texture pattern, and offers no stable and versatile boundary line detection processing scheme.
As a method of extracting information associated with an object by performing template matching, i.e., as a technique that can be used for searching for, tracking, or recognizing a specific object from an image sensed by a camera, a model base technique performed based on feature vector extraction (constituting line segment, shape parameter extraction) processing and subsequent comparison with a feature vector model of a feature vector is known (Japanese Patent Publication No. 6-14361, Japanese Patent Laid-Open No. 6-4673, and the like).
As a technique for detecting the motion of an object, a method disclosed in Japanese Patent Laid-Open No. 5-232908 cuts the portion to be subjected to motion extraction on the basis of the luminance level of a projection component to track motions at the respective points in the regions of interest in time-series images.
However, in the former example, since templates or models having different sizes must be prepared for a target image in correspondence with changes in size of a specific object in an image to perform matching in units of regions of the image, a very large memory capacity and a very long calculation time for feature vector extraction and matching with models are required.
Alternatively, the zooming parameter or the like of the image sensing system must be manually adjusted, so that the size of the target image becomes nearly equal to that of the model.
In the latter example, it is generally difficult to stably cut out an action extraction portion on the basis of the luminance level of a projection component. of an image. Also, after motions at the respective points are tracked, it is difficult to interpret the motions at the respective points as one action category by combining such motion information, except for a simple action.
Furthermore, in constructing an image sensing apparatus or system, since the image sensing means does not have any command communication means for externally controlling the image sensing mode upon extraction of an object or any image sensing parameter control function required upon sensing an image for object extraction, image sensing conditions optimal to image extraction cannot be set.
Therefore, the image sensing conditions cannot be optimally set in correspondence with image taking situations such as a change in illumination condition, the presence/absence of object motions, the presence/absence of motions of the image sensing means itself, and the like for the purpose of image extraction.
Japanese Patent Laid-Open No. 6-253197 above discloses a technique in which a stop control unit is set to obtain an appropriate average luminance upon sensing the background image, the current image is sensed using the same setting value as that for the background image, and a specific object image is extracted based on difference data between the two images.
However, again, an image sensing system cannot set optimal image sensing conditions to image extraction since an image sensing unit has neither a command communication control unit for appropriately controlling the image sensing mode from an external device upon extracting a specific object image nor a control function of the image sensing parameters required for sensing an image used for extracting a specific object image. Therefore, the image sensing conditions cannot be optimally set in correspondence with image taking situations such as changes in illumination conditions, the presence/absence of object motion, the presence/absence of motions of the image sensing unit itself, and the like.
When a specific object image is to be extracted by remote-controlling a camera, a communication control means, a communication system, control commands, and the like has not been established yet. In particular, optimal image sensing conditions such as the field angle, focusing, illumination conditions (the presence/absence of flash emission), and the like for a designated object cannot be automatically or interactively set.
For example, setting an optimal field angle is important for removing the unwanted background region as much as possible and for efficiently performing image extraction processing. However, such function cannot be realized since communication control and image sensing control systems for performing such setting operation between the camera and the terminal device have not been established yet.
According to one mode of the present invention, it is an object of the present invention to provide an apparatus and method that can perform high-speed search processing of a specific object using information associated with the image sensing mode upon sensing an object.
It is another object of the present invention to attain high-speed discrimination of the presence/absence of a known object in an image using limited image sensing mode parameters.
It is still another object of the present invention to reduce the calculation time required for discriminating the presence/absence of a specific object.
It is still another object of the present invention to further reduce the calculation time.
It is still another object of the present invention to attain high-speed presence/absence discrimination processing of a specific object on the basis of contour line data and to attain object cutting processing simultaneously with the presence/absence discrimination processing.
It is still another object of the present invention to reduce the time required for discriminating the presence/absence of a specific object on the basis of only contour line data, and to reduce the cutting calculation time.
It is still another object of the present invention to allow stable detection of a specific object even when a standard image and an actual image of an object have different sizes and shapes in the case of processing based on contour line data.
It is still another object of the present invention to attain high-speed object extraction with high reliability by utilizing image sensing information.
It is still another object of the present invention to attain high-speed separation processing of a specific object from the background by cutting an image inside a contour line utilizing image sensing mode parameters.
According to the present invention, since a specific object is detected by comparing it with a standard image which is initialized based on image sensing mode information used upon sensing an image of an object by an image sensing means, the specific object can be detected at high speed. Therefore, a desired image can be found at high speed, and an automatic image taking operation having a specific object at the center can be realized.
For example, since the initial size of the standard image is set on the basis of the object distance upon image sensing or the focal length parameter of the image sensing means, the size of an object region that may be present in the image can be appropriately set, and the calculation time required for discriminating the presence/absence of a specific object can be reduced. Hence, the presence/absence discrimination processing of a specific object can be performed for only a region near the object position upon image sensing, and the calculation time can be further reduced.
On the other hand, since the central position of the standard image is initialized on the basis of the visual axis direction upon image sensing, the presence/absence discrimination processing of a specific object can be performed while limiting the search processing range in the image database to a range near the object position upon image sensing, and the calculation time required for discriminating the presence/absence of a specific object can be greatly reduced.
According to another mode of the present invention, it is an object of the present invention to efficiently and stably extract a specific object image from a plurality of images having different image sensing conditions.
It is another object of the present invention to cope with an object that is changing its shape or moving and to realize high-speed image extraction and, more particularly, to attain image extraction using a plurality of images having different image sensing conditions under the condition of a constant magnification.
It is still another object of the present invention to realize stable convergence and high precision of active contour processing.
It is still another object of the present invention to allow stable extraction of initial contour data by a small calculation amount.
It is still another object of the present invention to allow stable image extraction by a small data amount while suppressing the influence of noise.
It is still another object of the present invention to allow image extraction with stably high extraction precision.
It is still another object of the present invention to allow high-speed, stable contour extraction and image extraction of a specific object by a small calculation amount while suppressing the influence of the image of the background object.
It is still another object of the present invention to automatically set an initial contour line with high precision by indicating one point inside the object to be extracted or near the boundary line (contour line) of the background using an indication means such as a mouse.
According to the present invention, an initial contour is set on the basis of the comparison result of a plurality of images having different image sensing conditions, and the background image region is removed on the basis of the initial contour. Hence, an image of a specific object can be stably extracted at high speed. Prior to extraction of a specific object by image processing, an approximate contour line (a boundary line from a background) of a specific object can be stably obtained at high speed from a plurality of images having different image sensing conditions, and the calculation time required for removing the residual background image region can be greatly shortened.
Since an initial contour approximate to the contour line of the object to be extracted in active contour processing is set on the basis of a plurality of images having different image sensing conditions, automatic extraction of a specific object and moving image extraction of a moving object can be performed at high speed. Hence, an object in motion can be extracted from a moving image by processing for deforming and moving an approximate contour line, and high-speed image extraction can be realized.
Since initial contour data extracted from a plurality of images having different image sensing conditions can be appropriately set by interactive modification, a specific object can be extracted with high precision, and stability of background removal processing and image extraction precision can be improved using a modification means of an approximate contour line.
For example, an approximate contour line can be stably obtained from two images having different focusing states, thus greatly improving the image extraction precision and processing speed.
According to still another mode of the present invention, it is an object of the present invention to provide an image extraction apparatus having a large margin for differences in image sensing condition in each respective images upon extracting a specific object by comparing a plurality of images.
It is another object of the present invention to identify the region to be extracted according to the above object using a simple image processing means.
It is still another object of the present invention to obtain a high-quality image of the object to be extracted.
It is still another object of the present invention to attain efficient processing such as image edit, modification, transmission operations, and the like.
It is still another object of the present invention to allow image extraction with an increased margin for camera shake upon image sensing, variations in position of an image sensing means, and for variations in posture.
It is still another object of the present invention to increase the margin for variations in magnification condition, focusing state, contrast, illumination condition, and the like.
It is still another object of the present invention to allow image extraction with a large margin for variations in image sensing condition.
According to the present invention, since image sensing conditions are recorded, and image data is converted on the basis of the recorded image sensing conditions, when a specific object is extracted by comparing a plurality of images, e.g., a registered image and the current image, different frame images in a moving image, and the like, the margin for different image sensing conditions of the respective images can be increased. Hence, when a specific object image is extracted from the background image, the object can be satisfactorily extracted irrespective of very small variations in position of the image sensing means owing to camera shake, differences in exposure condition, variations in gain of the sensor, and the like. The margin for variations in, e.g., illumination condition can be increased without using any model associated with the object to be extracted such as a color model.
Since image data of the background image is converted using an image including a specific object sensed under the same image sensing conditions as those used upon sensing the background image, the margin for variations in image sensing conditions and camera parameters can be increased on the basis of both the background image and the image including a specific object, and a high-quality object image can be extracted independently of the image sensing conditions used upon sensing the background image. Therefore, a high-quality in-focus image of the object to be cut can be output.
For example, since the image sensing conditions include an exposure amount, focusing state, and the presence/absence of flash emission, an image of a specific object can be extracted from a plurality of images sensed under different image sensing conditions, and hence, image extraction with a large margin for variations in magnification condition, focusing state, contrast, illumination condition, and the like can be performed.
According to still another mode of the present invention, it is an object of the present invention to provide a sensed image processing apparatus which stably extracts information of a specific object at high speed while saving the memory capacity irrespective of the distance between the object and the image sensing means and the size on the screen.
It is another object of the present invention to provide a sensed image processing apparatus which is hardly influenced by differences in observation angle of the object.
It is still another object of the present invention to provide a sensed image processing apparatus which allows easy extraction of the distance between the object and the image sensing means and the object region in the screen, can automatically set a proper size of the model to be subjected to matching, and can easily and accurately recognize and extract a specific object.
It is still another object of the present invention to provide a sensed image processing apparatus which can perform accurate object extraction calculations without being sensitively influenced by variations in object size, and the like.
It is still another object of the present invention to provide a sensed image processing apparatus which can extract motion information of an image sensing means from time-series images, and can stably extract an object image irrespective of motions or actions of an object.
It is still another object of the present invention to provide a sensed image processing apparatus which can absorb misalignment between the center of the template model and that of the object region, and can perform high-speed processing.
It is still another object of the present invention to provide a stable, high-precision sensed image processing apparatus which is not influenced by variations in background pattern.
It is still another object of the present invention to provide a stable., high-precision sensed image processing apparatus which can detect the object distance even in the dark.
According to the present invention, since a template size determination means which determines the size range of a template that can be reduced/enlarged on the basis of the output from an object distance detection means is arranged, high-speed information extraction of a specific object can be stably performed while saving the memory capacity irrespective of the distance between the object and the image sensing means or the size on the screen.
On the other hand, since a plurality of templates, which are generated on the basis of images, viewed from different view point positions or directions, of an identical object, are stored, information extraction of a specific object is hardly influenced by differences in view angle of the object.
For example, since the template size determination means determines the temperature size in a predetermined range on the basis of the object distance, size information of a specific object, and optical parameters of the image sensing means, a proper size of the model to be subjected to matching can be automatically set.
Also, information extraction of a specific object can be performed without being sensitively influenced by variations in object size.
According to still another mode of the present invention, it is an object of the present invention to attain high-speed object extraction and to allow control of the image sensing conditions required for obtaining a high-quality image to be extracted.
It is another object of the present invention to allow external control of the characteristics of the image sensing operation for object extraction, and to transfer the extracted image to a remote place.
It is still another object of the present invention to allow external control of the image sensing parameters for object extraction, and to transfer the extracted object image to a remote place.
It is still another object of the present invention to automatically set the image sensing conditions for object extraction and to perform automatic parameter control.
It is still another object of the present invention to improve the operability of an image input device upon object extraction.
It is still another object of the present invention to allow an image input device to perform internal pre-processing of an image required for precise object extraction, and to shorten the time required from the image sensing operation until the output operation of the extracted image.
It is still another object of the present invention to stably cut an image of a specific object from an arbitrary background irrespective of the illumination conditions.
It is still another object of the present invention to allow high-precision object extraction without being influenced by the illumination conditions, and to attain exposure amount control and image cutting control based thereon so as to realize high-image quality upon extracting a specific object.
It is still another object of the present invention to selectively use appropriate image sensing parameters so as to attain high image quality of a cut image and high-speed processing in correspondence with the illumination conditions and object distance upon image sensing.
It is still another object of the present invention to stably cut images at high speed in image sensing operations with a plurality of image sensing conditions.
It is still another object of the present invention to automatically select an appropriate image compression mode in correspondence with whether the image to be cut is a still image or a moving image.
It is still another object of the present invention to stably cut an image independently of whether or not the object in question is a moving object or whether or not the object is moving relative to the image sensing means.
It is still another object of the present invention to appropriately attain an image sensing operation for object extraction without any control from an external terminal via a communication means or any complicated manual operation.
According to the present invention, when an image sensing means performs an image sensing operation for extracting a specific object image, it can set or control optimal image sensing parameters. Since image quality of the extracted image can be optimized by controlling the image sensing conditions, a specific object image can be extracted at high speed and with high image quality.
Since the feature amount or the like of an image can be externally controlled, the image sensing operation for object extraction and image processing to be performed inside an image input device can be remote-controlled, and the extracted image can be on-line transferred to a remote place.
For example, since the image sensing parameters can be controlled by externally supplying control commands via a control command communication means, the image sensing parameters for object extraction can be externally controlled, and high image quality of the extracted image and high-speed extraction processing can be realized.
Since an image sensing mode setting means has an image setting mode setting button, the image sensing mode setting operation is started upon operation of the image sensing mode setting button, and the control amounts of image sensing parameters are set on the basis of the image sensing parameter measurement values, the image sensing parameters in the object extraction mode can be automatically set, and the control amounts of image sensing parameters need not be manually set.
Other features and advantages of the present invention will be apparent from the following description taken in conjunction with the accompanying drawings, in which like reference characters designate the same or similar parts throughout the figures thereof.
According to still another mode of the present invention, it is an object of the present invention to provide an image sensing apparatus and method, which can selectively control image sensing operations for extraction, recognition, tracking, and the like of a specific object by either a remote-control operation by means of communication control from an external terminal device or an operation on the camera.
It is another object of the present invention to provide an image sensing apparatus and method, which can set appropriate image sensing conditions having an object as the center on the basis of image data when an image sensing operation for object extraction is performed by a remote-control operation.
It is still another object of the present invention to provide an image sensing apparatus and method, which can optimize and automate the image sensing system for object extraction on the basis of image data.
It is still another object of the present invention to provide an image sensing apparatus and method that can perform an image sensing (image fetching) operation for object extraction at an arbitrary time and timing from the external terminal device side.
It is still another object of the present invention to provide an image sensing apparatus and method which can attain high-speed processing from an image sensing operation to object extraction, and high-speed display processing on a display of a terminal device.
It is still another object of the present invention to provide an image sensing apparatus and method which allow the terminal device side to control a series of processing operations from the setting operation of image sensing conditions to the display operation of the extracted image.
It is still another object of the present invention to provide an image sensing apparatus and method which can remote-control the image sensing parameters for object extraction in a cordless manner.
It is still another object of the present invention to provide an image sensing apparatus and method which can externally supply an image sensing operation control program recorded on a recording medium, and can automatically recover an identical image sensing operation for object extraction (the processing sequence in the camera).
It is still another object of the present invention to provide an image sensing apparatus and method which can externally supply image sensing operation control data recorded on a recording medium, and can automatically recover identical image sensing conditions for object extraction.
It is still another object of the present invention to provide an image sensing apparatus and method which can automatically recover an identical image sensing operation later when an image sensing operation for object extraction is manually performed.
According to the present invention, in image sensing means, image sensing parameter control means controls image sensing parameters, and image sensing mode setting means sets an image sensing mode. In calculation means, first image processing means calculates comparison data between a plurality of images with the controlled image sensing parameters, and second image processing means classifies image regions by discriminating the calculated comparison data. In terminal means, data communication means performs data communications with the image sensing means, and the image sensing parameter control means controls the image sensing parameters on the basis of a signal from the terminal means or a signal from the image sensing mode setting means. Therefore, image sensing operations for extraction, recognition, tracking, and the like of an object can be selectively attained by a remote-control operation by means of communication control from an external terminal device or an operation on the image sensing means (camera).
Alternatively, in image sensing means, image sensing parameter control means controls image sensing parameters, and image processing means calculates comparison data between a plurality of images with the controlled image sensing parameters. In calculation means, indication/selection means indicates an object position in the image. When terminal means performs data communications with the image sensing means, the calculation means generates an image sensing parameter control signal on the basis of the object position signal from the indication/selection means and image data in a predetermined size region having the indicated position as the center, the image sensing parameter control means controls image sensing conditions on the basis of the image sensing parameter control signal, and the terminal means extracts a specific object image by processing an image obtained under the controlled image sensing conditions. Hence, appropriate image sensing conditions having an object as the center can be set on the basis of image data.
Alternatively, an apparatus comprises image sensing means having finder display means, image display means for displaying an image sensed by the image sensing means, indication/selection means for indicating the image displayed on the image display means, and calculation means having region discrimination processing means for discriminating an indicated region of the image. The region discrimination processing means processes image data sensed by the image sensing means and performs cutting processing having a position or region in an image frame output from the indication/selection means as the center, and the image display means or the finder display means displays the extracted image signal from the calculation means. As a consequence, high-speed processing from an image sensing operation to object extraction and high-speed display processing on a camera or a display of a terminal device can be realized.
Alternatively, an apparatus comprises image sensing means having image sensing parameter control means for controlling image sensing parameters and memory means for storing sensed image data, image display means for displaying an image sensed by the image sensing means, indication/selection means for indicating the displayed image, calculation means having region discrimination means for discriminating a region of the indicated image, and terminal means having data communication means for performing data communications with the image sensing means. The image sensing parameter control means receives control data from the terminal means via the data communication means or the memory means. The region discrimination processing means calculates comparison data between a plurality of images sensed under at least two image sensing conditions including a standard image sensing condition and a non-standard image sensing condition set by the image sensing parameter control means, and performs cutting processing having a position or region in an image frame output from the indication/selection means as the center. The image display means displays an image signal output from the calculation means. Therefore, a series of processing operations from the setting operation of the image sensing conditions to the display operation of the extracted image can be controlled by the terminal device side.
Alternatively, a method comprises the image sensing step including the image sensing parameter control step of controlling image sensing parameters of an image sensing apparatus and the image sensing mode setting step of setting an image sensing mode, the calculation step including the first image processing step of calculating comparison data between a plurality of images with the controlled image sensing parameters and the second image processing step of classifying image regions by discriminating the calculated comparison data, and the data communication step of performing data communications between the image sensing apparatus and a terminal device. Since the image sensing parameter control step controls the image sensing parameters on the basis of a signal from the terminal device or a signal set in the image sensing mode setting step, image sensing operations for extraction, recognition, tracking, and the like of an object can be selectively attained by a remote-control operation by means of communication control from an external terminal device or an operation on image sensing means (camera).
Alternatively, a sensed image processing method comprises the image sensing step including the image sensing parameter control step of controlling image sensing parameters of an image sensing apparatus and the image processing step of calculating comparison data between a plurality of images with the controlled image sensing parameters, the calculation step of indicating an object position in the image using an indication/selection device, and the data communication step of performing data communications between the image sensing apparatus and a terminal device. The calculation step generates an image sensing parameter control signal on the basis of an object position signal from the indication/selection device and image data in a predetermined size region having the indicated position as the center, the image sensing parameter control step controls image sensing conditions on the basis of the image sensing parameter control signal, and the terminal device extracts a specific object image by processing an image obtained under the image sensing conditions. Accordingly, appropriate image sensing conditions having an object as the center can be set on the basis of image data.
Other features and advantages of the present invention will be apparent from the following description taken in conjunction with the accompanying drawings, in which like reference characters designate the same or similar parts throughout the figures thereof.